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Abstract — Based on the extended binary image of non-binary 
LDPC codes, we propose a method for generating extra redun- 
dant bits, such as to decreases the coding rate of a mother 
code. The proposed method allows for using the same decoder, 
regardless of how many extra redundant bits have been produced, 
which considerably increases the flexibility of the system without 
significantly increasing its complexity. Extended codes are also 
optimized for the binary erasure channel, by using density 
evolution methods. Nevertheless, the results presented in this 
paper can easily be extrapolated to more general channel models. 

Index Terms — Non-binary LDPC codes, extended binary im- 
age, incremental redundancy, very small coding rates. 

I. Introduction 

Adaptive coding techniques are frequently employed, espe- 
cially in wireless communications, in order to dynamically 
adjust the coding rate to changing channel conditions. An 
example of adaptive coding technique consists in puncturing 
a mother code. When the channel conditions are good more 
bits are punctured and the coding rate is increased. In poor 
channel conditions all redundant bits are transmitted and the 
coding rate drops. However, in harsh conditions, the receiver 
might not be able to successfully decode the received signal, 
even if all the redundant bits have been transmitted. In such 
a case, the coded block can be retransmitted until the sent 
information is successfully decoded. This is equivalent to 
additional repetition coding, which further lowers the coding 
rate below the mother coding rate. 

However, the use of retransmission techniques might not 
be suitable nor possible in some situations, such as multi- 
cast/broadcast transmissions, or whenever the return link is 
strictly limited or not available (such situations are generally 
encountered in satellite communications). The main alternative 
in this case is the use of erasure codes that operate at 
the transport or the application layer of the communication 
system: source data packets are extended with redundant (also 
refeiTed to as repair) packets that are used to recover the lost 
data at the receiver. Physical (PHY) and upper layer (UL) 
codes are not mutually exclusive, but they are complementary 
to each other. Adaptive coding schemes are also required at 
the upper layer, in order to dynamically adjust to variable loss 
rates. Besides, codes with very small rates or even rateless IT], 
llll are sometimes used at the application layer for fountain- 
like content distribution applications. 
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In this paper we propose a coding technique that allows to 
produce extra redundant bits, such as to decrease the coding 
rate below the mother coding rate. Extra redundant bits can be 
produced in an incremental way, yielding very small coding 
rates, or can be optimized for a given target rate below the 
mother coding rate. As for puncturing, the proposed technique 
allows for using the same decoder, regardless of how many 
extra redundant bits have been produced, which considerably 
increases the flexibility of the system, without increasing its 
complexity. 

The proposed coding scheme is based on non-binary low 
density parity check (NB-LDPC) codes f3l or, more precisely, 
on their extended binary image [4|. If q = 2^ denotes 
the size of the non-binary alphabet, each non-binary symbol 
corresponds to a p-tuple of bits, referred to as its binary image. 
Extra redundant bits, called extended bits, are generated as the 
XOR of some bits from the binary image of the same non- 
binary coded symbol. If a certain number of extended bits are 
transmitted over the channel, we obtain an extended code, the 
coding rate of which is referred to as extended (coding) rate. 
In the extreme case when all the extended bits are transmitted, 
the mother code is turned into a very small rate code, and can 
be used for fountain-like content distribution applications flU. 
A similar approach to fountain codes, by using multiplicatively 
repeated NB-LDPC codes, has been proposed in [5|. If some 
extended rate is targeted, we show that the extended code can 
be optimized by using density evolution methods. 

The paper is organized as follows. Section |II]gives the basic 
definitions and the notation related to NB-LDPC codes. In 
Section |III1 we introduce the extended NB-LDPC codes and 
discuss their erasure decoding. The analysis and optimization 
of extended NB-LDPC codes are addressed in Section |IV] 
Section |V] focuses on the code design and presents simulation 
results, and Section [VTl concludes the paper. 

II. NoN-BiNARY LDPC Codes 

We consider NB-LDPC codes defined over the finite 

field with q elements, where q = 2^" is a power of 2 (this 
condition is only assumed for practical reasons). We fix once 
for all an isomorphism of vector spaces; 



F„ 



Elements of Fg will also be referred to as symbols, and 
we say that x — (xq, . . . , a;p_i) G F2 is the binary image 
of the symbol X e Fg, if they correspond to each other 



by the above isomorphism. A non-binary LDPC code over 
¥g is defined as the kernel of a sparse parity-check matrix 
H E MA/jv(Fg). Ahernatively, it can be represented by 
a bipartite (Tanner) graph [7| containing symbol-nodes and 
constraint-nodes associated respectively with the N columns 
and M rows of H. A symbol-node and a constraint-node are 
connected by an edge if and only if the corresponding entry of 
H is non-zero; in this case, the edge is assumed to be labeled 
by the non-zero entry. As usually [8J, we denote by A and p 
the left (symbol) and right (constraint) edge-perspective degree 
distribution polynomials. Hence, X{x) = X^d^'i^''"^ ^^'^ 
p{^) — J^d Pd.x'^^^' where and represent the fraction 
of edges connected respectively to symbol and constraint 
nodes of degree-d. The design coding rate is defined as 

So p{x)dx 



1 



and it is equal to the coding rate if and 



only if the parity-check matrix is full-rank. 

III. Extended Non-binary LDPC Codes 

A. Extended code description 

For any integer 1 < A; < (7 — 1, let [/c] = (/cq, • • • , fcp-i)^ 
denote the column vector corresponding to the binary decom- 
position of fc; i.e. k = ^^Z^ ki2\ with ki € {0, 1}. Let 
X E ¥q he a non-binary symbol, and x = {xo, • • • , a^p-i) be 
its binary image. The k-th extended bit of X is by definition: 



ak — x^ [k] 



p-i 

i=0 



The vector a = (ai, . . . , a,j_i) is called extended binary 
image of X. Note that = Xi, for any < i < p — 1. 
An extended bit is said to be nontrivial if k is not a power 
of 2 (hence, is a linear combination of at least two bits 
from the binary image x of X). 

Now, consider a NB-LDPC code defined over F^, with 
coding rate r ^ j^. Let (Xi,...,Xjv) G be a non- 



binary codeword, {x^, - . ■ ^x^j) G ^ be its binary image, 
and {oLiT ■ ■ tQ.n) G F^^'^ be its extended binary image. 
By transmitting the extended binary image over the channel, 
we obtain a code with rate ^r!^^ t\ — r-^, which can 
be advantageously used for application requiring very small 
coding rates |4|. 

We define an extension of the NB-LDPC code, as a family 
of matrices {Ai, . . . , Am}, where each An £ Mp^t^(F2) is a 
binary matrix with p rows and t„ columns, with tn > 0. Let 
Qi.n= 3in^ £ Fj" ; hcncc, a„ is constituted of t„ extended 
bits of Xn (possibly with repetitions, if An contains two or 
more identical columns). The binary vector (fli, • ■ • , a^v) is 
called extended codeword, and the extended coding rate is 
given by = where T = Y.n=i 

Note that the above definition is very broad, and it can 
yield extended rates below as well as extended rates above the 
mother coding rate. In particular, it includes punctured codes: 
if An = col([2i],...,[2P-i]), then a„ = (a;„,i, . . . , x„,p_i), 
which is the same as puncturing the first bit, Xn.o, from 
the binary image of X„. Moreover, taking some i„ = is 




Figure 1. Extension of a NB-LDPC code. Blue circles represent bits of the 
binary image (p = 3), while red circles represent (nontrivial) extended bits. 



equivalent to puncturing the whole symbol Xn- The optimiza- 
tion of puncturing distributions for NB-LDPC codes has been 
addressed in |9 |. In this paper, we restrict ourselves to the case 
when matrices An are of the form An = [Ip \ Bn], where Ip 
is the p X p identity matrix, meaning that each a„ contains 
the binary image Xn (the use of "extension" complies with its 
literal meaning). We will further assume that any two columns 
of a matrix An are different. It follows that p < tn < 5 — 1, 
and each a„ is constituted of the binary image x„ and <„ — p 
pairwise different (nontrivial) extended bits. In this case, we 
shall say that the symbol Xn is extended by k = tn — p bits. 

For instance, let p = 3 and consider that the following 
matrix A is used to extend by 2 bits some coded symbol X: 



XX A, where A 



10 10 
10 1 
111 



Then a = (ai, a2, a^, 0:5, ae) = {xo,xi,X2, xq Ax2,xi AX2), 
where A denotes the bit-xor operator 

In order to determine the coding rate of the extended code, 
we denote by fc the fraction of degree-d symbols with k 
nontrivial extended bits, < fc < q — p', thus X]fe=o ^ fd,k~^- 
The average number of nontrivial extended bits per coded 
symbol is given by / = Ed=i ELo"^ ^^/d^fe, where ds 
is the maximum symbol node degree, and = 



we can 
r-^, r 

q-l ' 



d f^^ \(x)dx 

is the fraction of degree-d symbol nodes. It follows that the 
extended coding rate is given by = f^^^- Thus, 

achieve an arbitrary extended rate within the interval 
by varying the parameter /. 

Figure [T] illustrates an extended code defined over Fs, with 

/2,o = /2,i = /2.2 = /2.4 = 1/4, and /a, 2 = /3,3 = 1/2, 
which correspond to / = 2. The mother coding rate is r = 0.5 
and the extended coding rate is rg — 0.3. 

B. Iterative erasure decoding 

We consider that the extended codeword (oi, . . . , a^v) is 
transmitted over a binary erasure channel (BEC). At the 
receiver part, the received bits (both from the binary image and 
extended bits) are used to reconstruct the corresponding non- 
binary symbols. Precisely, for each received bit we know its 
position within the extended binary image of the correspond- 
ing symbol. Hence, for each symbol node we can determine 
a set of eligible symbols that is constituted of symbols whose 
extended binary images match the received bits. These sets 



are then iteratively updated, according to the linear constraints 
between symbol-nodes fTOl. Alternatively (and equivalently), 
the extended code can be decoded by using the linear-time 
erasure decoding proposed in [4 |. 

The asymptotic threshold of an ensemble of codes is defined 
as the maximum erasure probability pth that allows transmis- 
sion with an arbitrary small error probability when the code 
length tends to infinity fSl. Given an ensemble of codes, its 
threshold value can be efficiently computed by tracking the 
fraction of erased messages passed during the belief propaga- 
tion decoding; this method is referred to as density evolution. 
In this paper, the density evolution is approximated based 
on the Monte-Carlo simulation of an infinite code, similar 
to the method presented in [9|. This method has two main 
advantages: it can easily incorporate the extending distribution 
{fd.k}d.k, and it can be extrapolated to more general channel 
models. 

IV. Analysis and optimization 

The goal of this section is to answer the following questions. 

First of all, assume that we have given a symbol node that 
has to be extended by k bits. How should they be chosen 
among the q — p — 1 (nontrivial) extended bits? 

Secondly, given an extended coding rate r^, how should be 
extended bits distributed over the symbol-nodes? Put differ- 
ently, which is the optimal extending distribution {/d,fc}? 

A. Extended bits selection strategy 

We assume that we have given a symbol-node that has to 
be extended by k bits. A choice of the k bits among the 
q — p — 1 extended bits corresponds to an extending matrix 
A — [Ip \ B] of size p x (p + k), with pairwise distinct 
columns. For each such a matrix, assume that the extended 
symbol a — xxAis transmitted over the BEC, and let E{A) 
be the expected number of eligible symbols at the receiver 
Recall that an eligible symbol is a symbol whose extended 
binary image match the received bits. If all transmitted bits 
have been erased, any symbol is eligible. Conversely, if the 
received bits completely determine the non-binary symbol. 
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Figure 2. 1-bit extension for regular NB-LDPC codes over Fig 



then there is only one eligible symbol. More generally, let 
Ajg;, denote the sequence of received bits, and A^^c denote the 
submatrix of A determined by the columns that correspond 
to the received positions of a. Then the eligible symbols are 
the solutions of the linear system x x Ai-e^ ~ > ^^d their 
number is equal to 2^^™''^"*"'^). Now, if e denotes the erasure 
probability of the BEC, it can be easily verified that: 



E{A) 



p+k 

E 

i=0 



(1 - eyeP+'" 




where the second sum takes over all the submatrices Ai cons- 
tituted of i among the p + k columns of A. Hence, in order 
to minimize the expected number of eligible symbols E{A), 
we choose A such that (imin(j4) is maximal, where dini„{A) is 
the smallest number of linearly dependent columns of A. 

1) One extended bit per symbol node: Consider the en- 
semble of regular (A(a;) ~ x, p{x) = x^^ LDPC codes defined 
over the Fig. Assume that each symbol-node is extended by 
k = 1 bit, such as to achieve an extended rate = 0.4. 
According to the choice of the extended bit (among the 11 
nontrivial extended bits), dmin may be equal to 3,4, or 5. 
The asymptotic threshold corresponding to each choice of the 
extended bit is shown in Figure |2] Note that extended bits are 
ordered on the abscissa according to the corresponding d^-nm- 
For comparison purposes, we show also the asymptotic thresh- 
old corresponding to the repetition of some bit from the binary 
image (trivial extended bit — Xi), in which case d^-nm = 2. 
Also, the blue line correspond to the threshold obtained 
if each symbol node was extended by choosing a random 
nontrivial extended bit. We observe that the best threshold is 
obtained when each symbol node is extended by 0:15, which is 
the XOR of the four bits xq,xi,X2, X3 of the binary image. 

2) Several extended bits per symbol node: We consider 
two ensembles of regular codes Ci (A(x) = x,p{x) — a;"^), 
C2 (A(a;) = x'^, p{x) = x^) and one ensemble of semi-regular 
codes C3 (A(x) = 0.5a; + 0.5x'*,p(x) = 0.25a;'' + 0.75a;5), of 
coding rate r — 1/2, defined over Fig. For each ensemble of 
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Figure 3. fc-bits extension for regular and semi-regular NB-LDPC codes 
over F16 
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Figure 4. Comparison of spreading vs. clustering extending distributions for 
regular NB-LDPC codes over Fis 



codes, we consider five different cases, in which all symbol 
nodes are extended by the same number fc of bits, with fc = 1, 
2, 3, 4, and 5. Accordingly, the extended coding rate = 0.4, 
0.33, 0.29, 0.25, and 0.22. 

The normalized gap to capacity, defined as: 



capacity — threshold 1 



Pth 



capacity 



1 - r 



is shown in Figure |3] Solid curves correspond to a dmin- 
optimized choice of the extended bits, while dashed curves 
correspond to a random choice of the extended bits. For k — 5, 
there is only a small difference between these two strategies. 
However, when fc < 4, the gain of the dmin-optimized choice 
is significant for both regular and semi-regular codes. 

B. Extending distribution analysis 

First of all, we discuss the case of regular codes. In Figure 2] 
we consider three ensembles of regular codes over Fig, with 
coding rate r = 0.5. For each ensemble of codes, we consider 
five cases, corresponding to values of k between 1 and 5. In 
each case, a fraction fk of symbol-nodes are extended by k 
bits, while the remaining symbol-nodes have no extended bit. 
The fraction fk is chosen such that the extended coding rate 
re = 0.4. Hence, fk = 1,0.5,0.33,0.25,0.20,0.09, for k = 
1, 2, 3, 4, 5, respectively. The right most point on the abscissa 
corresponds to a sixth case, in which the extended rate — 

0. 4 is achieved by extending 9% of symbol-nodes by fc = 11 
bits (hence, k — q — p ~ 1, which is the maximum number 
of extended bits). For any of the three ensembles, we can 
observe that the smallest gap to capacity is obtained for k = 

1, which means that extended bits are spread over as many 
symbol nodes as possible (in this case, 100%), instead of being 
clustered over the smallest possible number of symbol-nodes. 

In case of irregular NB-LDPC codes, let (f) — {/d,fc}^ j, be 
an extending distribution. Thus, fd^k is the fraction of degree-d 
symbols with fc nontrivial extended bits, < fc < {q~p). Let 
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Figure 5. Comparison of several extending -distiibutions, for semi-regular 
NB-LDPC codes over Fie 



fd denote the average number of extended bits per symbol- 
node of degree-d; that is: 



q-p-l 

fd= Yl ^fd^^ ^ [0,g-p- 1] 



We say that the extending distribution (f) is of sp reading-type 
if for any degree fd^k ^ only if A; = [fd\ or k = \fd]. 
In different words, for any degree d, the extended bits are 
uniformly spread over all the symbol-nodes of degree d. 
Clearly, a spreading-type distribution is completely determined 
by the parameters {fd}, as we have fd^lf^] = \fd\ ~ fd, 
fd.lM - lfd\ - fd, and fd^k = for A: ^ {[fd\. \fd]}. 

We say that the extending distribution <f> is of clustering-type 
if for any degree d, fd.k 7^ only if fc — q — p+1. In different 
words, for any degree d, the extended bits are clustered over 
the smallest possible fraction of symbol-nodes of degree d. 
Clearly, a clustering-type distribution is completely determined 
by the parameters {fd}, as we have fd,q-p+i = gjp^i and 
fd.k^O for kj^q-p+ 1. 

Now, let us consider the ensemble of semi-regular LDPC 
codes over Fig with edge-perspective degree distribution poly- 
nomials X{x) = 0.5a; + 0.5a;'' and p{x) = 0.25a;'' + 0.75a;^. 
The mother coding rate is r = 0.5, and we intend to 
extend symbol-nodes such as to achieve extended coding rates 
e {0.45,0.4,0.35,0.3}. Several extending distributions 
are compared in Figure |5] There are three spreading-type 
distributions, which spread the extended bits over all the 
symbol-nodes, or only over the symbol-nodes of degree either 
2 or 5, and two clustering-type distributions, which cluster the 
extended bits over the symbol-nodes of degree either 2 or 5. 
In all cases, extended bits (or, equivalently, extending matrices 
An) are chosen such as to maximize the corresponding dmin 
values. We observe that the smallest gap to capacity is obtained 
for extending distributions that spread extended bits either over 
the degree-5 symbol nodes only (rg = 0.45, 0.4), or over all 
the symbol-nodes (re = 0.35, 0.3). 



Table I 

Optimized extending distributions for a motherNB-LDPC code 
WITH r = 0.5 over Fi6, for re = {0.45 , 0.4, 0.35,0.3, 0.25, 0.2} 
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C. Extending distribution optimization 

Based on the above analysis, we only consider spreading- 
type extending distributions. Such an extending distribution 
is completely determined by the parameters {/d}, and the 
extended coding rate can be computed by = i ^ . , 

where is the fraction of symbol-nodes of degree d. 

For given degree distribution polynomials A and p, and 
a given extending rate r^, we use the differential evolution 
algorithm IfTTI to search for parameters {f^} that minimize 
the asymptotic gap to capacity. We assume that, for each 
symbol-node, the extended bits are chosen such as to maximize 
the corresponding dmin- The optimized extended codes are 
presented in the next section. 

V. Code Design and Performance 

In this section we present optimized extending distributions 
for an in^egular mother code over Fig. The mother code has 
coding rate r ^ 1/2, and it has been optimized by density 
evolution. The asymptotic threshold is pth ~ 0.4945, and the 
edge-perspective degree distribution polynomials are: 

X{x) = 0.596a; + O.lSGx^ + O.OTLt^ + 0.147x" 
p{x) = 0.2836a;'* + 0.7164a;5 

We optimized extending distributions for extended rates 
re G {0.45,0.40,0.35,0.30,0.25,0.20}. Optimized distribu- 
tions {fd} are shown in Table H] together with the correspond- 
ing asymptotic threshold pth and normalized gap to capacity 5. 
For comparison purposes, we have also indicated the normal- 
ized gap to capacity Sr^nd corresponding to a random choice 
of extended bits. The last column corresponds to extended 
rate = 2/15, obtained by extending each symbol-node by 
the maximum number of extended bits, i.e. q — p — 1 — 11 
bits. It can be observed that the optimized distributions allow 
to maintain an almost constant value of (5 « 0.01, for all 
extended rates 0.45 > r-g > 2/15. 

Finally, Figure |6] presents the Bit Erasure Rate (BER) 
performance of optimized extending distributions for finite 
code lengths. All the codes have binary dimension (number 
of source bits) equal to 5000 bits (1250 Fig-symbols). The 
mother code with rate 1/2 has been constructed by using 
the Progressive Edge Growth (PEG) algorithm (12], and 
symbol nodes have been extended according to the optimized 
distributions (extension matrices An being chosen such as to 
maximize (imin(^n))- 
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Figure 6. Finite length performance of optimized extended NB-LDPC codes 

VI. Conclusions 

Based on the extended binary image of NB-LDPC codes, 
we presented a coding technique that allows to produce extra 
redundant bits, such as to decreases the coding rate of a mother 
code. The proposed method allows for using the same decoder 
as for the mother code: extra redundant bits transmitted over 
the channel are only used to "improve the quality of the 
decoder input". 

Extending distributions for regular and irregular codes have 
been analyzed by using simulated density evolution thresholds 
of extended codes over the BEC. We have also presented 
optimized extending distributions, which exhibit a normalized 
gap to capacity 5 k, 0.01, for extended rates from 0.45 to 2/15 

Finally, although this paper dealt only with NB-LDPC 
codes over the BEC, the results presented here can be easily 
generalized to different channel models. 
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